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Data Use Multi-State Spotlight: 
Using Data Fidelity Tools to 
Improve Data Quality 


Challenge: How can states use fidelity tools to improve 
data quality? 


‘To evaluate interim progress toward the State-identified Measurable Result (SIMR), 
states require access to high-quality data from local education agencies (LEAs) and 
‘arly intervention service praviders. In a review of 2017 Phase III State Systemic 
Impravernent Plans (SIP), 43 Part C states noted limitations or concerns related to 
data and specifically data qualty. Meanwhile, 36 Part B states reported efforts to 
address fidalty by providing a method for collection and use of data regarding 
practice implementation. In turn, cancers about the quality of local data create 
barriers to effectively evaluating the implementation of the SSIP and progress 
toward the SIMR. Increasing fidelity to the data collection process can improve data 
‘quality. This spotlight shares how three states used three different appraaches ta 
data fidelity tools to improve data quality. It concludes with recammendations for 
States facing similar challenges. 


Use publisher fidelity tools built into the 
statewide assessment system. 


vom |B The 1owa Department of Education (IDE) aimed to increase 
adoption and use ofa state-provided literacy screening and 
progress monitoring toal by LEAs to scale up Towa’s multi-tiered system of 
‘supports. Iowa selected the Formative Assessment System for Teachers (FAST; 
ww fastbridge org) as ther state-identiiee assessment tool and provided i at na 
‘ost fo LEAS who chose to use FAST. Although IDE does not require LEAS or school 
staff to follow specific fideity procedures, the state selected an assessment system 
that provided fidelity tools for each of the measures and bult in certification 
procedures fer each measure. Staff can complete certification quizzes prior to 
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‘administering assessments and as a refresher from year to year. In adlion to the 
publisher-previded materials, IDE provides large-scale, in-person trainings each 
Summer. These trainings emphasized that obtaining reliable and valid student data 
is aentical component for making intervention decisions about students. 


Use existing fidelity tools from national technical 
assistance centers. 
| “The wyoming Department of Education (WDE) i implementing 


"st data-based Individualizaton (DBI) as one af its coherent 
Improvement strategies to improve Iteracy outcomes of students with disabilities 
‘The National Center on Intensive Intervention (NCI; 

ww. intensiveintervention.org) has identified DBI as a research-based process for 
inalvidualzing and intensiying interventions through the systematic use of 
‘assessment data, validated interventions, and research-based adaptation 
‘strategies. WDE adopted the data fidelity tools published by NCIIto increase data 
— at ail levels evaluation, including implementation and impact on student 
performance, Ongoing professional development and coaching are provided to 
teachers to ensure data are entered accurately and on time during initial, 
implementation. Descriptions of each data collection tool and its use are given next. 


+ NCI DBI Implementation Rubric and 
Interview. Schools complete an 
[adaptation of the S-point scale rubric 
— the guidance ofa trained DE 
‘cach. The data are used by the coach to 
‘Quide professional development and 
coaching supports for the teacher and are 
submitted directly tothe state's external 
‘evaluator to monitar changes in implementation 


‘The data use team responds to 


technical assistance (TA) 
requests that states submit ta 
theirTA proviger 


+ NCL DBT Implementation Log: Dally and Weekly Intervention Review. 
Teachers complete the electronic og each day af intervention delivery. At the 
tend of each Week, the DBI coach and the participating teachers use the data 
to problem salve around challenges with fidelity of implementation and 
student progress. While intended to guide decisions about student-level 
instruction and intervention adapations, the progress monitoring data are 
‘aggragated atthe state level ta document impact of DBI on student growth, 
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Create a new data quality tool. 


“The Colorado Office of Early Childhood (CEC) aimed to improve 
the quality of information included in Individual Family Service 
Plans (IFSP). With this goal in mind, the OEC created a 
comprehensive tool to standardize the process for reviewing IFSPs ta Improve 
{amity and child outcomes and to improve evaluation outcomes. Colorado used the 
following strategies to develop a tool and process for conducting standardized 
reviews of IFSPs: 


+ Created a web-based data system. The state created a new web-based data 
system that colcted not enly compliance data but also all the information 
contained inthe IFSP. 

+ Developed a customized data quality measurement tool. Colorado adapted 
the Kansas Infant-Toddler Services Quality Indicator Rubric to develop the 
‘Quality IFSP Outcame (QIO) tool, The state customized the QlO tool to 
match the information captured inthe Colorada TFSP documents. 

‘+ Gathered feedback. OEC engaged kay stakeholders in an iterative feedback 
Toop during development. OEC also conducted intemal texting and piloted 
scaring of the QIO toa 

‘+ Provided technical assistance (TA). OEC pravided TA to pilot sites through 
manthly phone calls, webinars, and 2 QUO User Manual. 


Each month, providers conduct approximately 110 self-assessments and OEC staff 
‘verify approximately 85 self-assessments using the QIO toal. Preliminary findings 
‘suggest that using the QIO tool has positively impacted the SSIP implementation. 
For example, high ratings on the QIO tool suggest increased collaboration between 
family members and service providers. Increased collaboration may lead to gathering 
Fieher information about the child's needs, which in turn, ean generate better results 
{or families. In addition, the QIO also provides information on the fdelty f desired 
practices and impact on the Improvement in child outcomes, 


Towa WYOMING ‘COLORADO 
Want to ead mare about Click here to learn more Interested in learning 
Towa's use ofa statewide about Project WIN and mare about the QIO Tool? 
assessment system? Click view Wyoming's adapted Click here for more 

here. resources and tools, information, 
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Recommendations for States Facing Similar Challenges 


'+ Communicate a clear and consistent message to LEAs and service providers 
— the importance of collecting high-quality and rallable data. 

‘+ Review preexisting data quality measurement tools that may be adapted to 
‘create a tool that meets the needs of the local context and data source 

+ Engage stakeholders in the data quality tool selection or development. This 
promates buy-in from the communities that will utiize the tool 

+ Captalize on widely available resources through national TA centers, 

1 Leverage fidelty tools and certification available through a publisher when 
using an assessment system. 

‘+ Regularly share data on quality with internal and external stakeholders, 


‘Available Resources 

+ ‘National Center for Systemic Improvement (NCSI), Technical Assistance 
‘State Facilitators (Find your state on the map.) 

+ NCSI Data Use Team Technical Assistance Support (Contact: Kristin Ruedel 
— kruedeliai.ora) 

+The Center far IDEA Early Childhood Data Systems, Collecting High-Quelity 
‘Child Outcome Data for Program Improvement Presentation 

+ National Center on Intensive Intervention Fidelity Resources, 


About this resource: This resource was developed by members ofthe NCSI Data Use 
‘Service Area Team, Incuding Kristin Ruedel (AIR), Gena Nelson (AIR), and Tessie Bley 
(AIR) —— 
— — 

WDE. The content was developed under cooperative agreement 

— ##1326R140005 (NCSI) from the Office of Special Eaucation ragrams, US. 
Department of Education. Opinions expressed herein donot necessarily represent the policy 
fof the U.S. Department of Education, and you should not assume endersement by he 
federal government. Project Officers: Pery Willams and Shedeh Hajghasseral 
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